Plant Seedlings Classification - Computer Vision Project¶
| Project Submission | Date |
|---|---|
| Rob Barker | October 12, 2024 |
| Filename | RobBarker_CV_PlantSeeding_FC.html/ipynb |
| Course | Computer Vision (CNN) |
Problem Statement¶
Context¶
In recent times, the field of agriculture has been in urgent need of modernizing, since the amount of manual work people need to put in to check if plants are growing correctly is still highly extensive. Despite several advances in agricultural technology, people working in the agricultural industry still need to have the ability to sort and recognize different plants and weeds, which takes a lot of time and effort in the long term. The potential is ripe for this trillion-dollar industry to be greatly impacted by technological innovations that cut down on the requirement for manual labor, and this is where Artificial Intelligence can actually benefit the workers in this field, as the time and energy required to identify plant seedlings will be greatly shortened by the use of AI and Deep Learning. The ability to do so far more efficiently and even more effectively than experienced manual labor, could lead to better crop yields, the freeing up of human inolvement for higher-order agricultural decision making, and in the long term will result in more sustainable environmental practices in agriculture as well.
Objective¶
The aim of this project is to Build a Convolutional Neural Network to classify plant seedlings into their respective categories.
Data Dictionary¶
The Aarhus University Signal Processing group, in collaboration with the University of Southern Denmark, has recently released a dataset containing images of unique plants belonging to 12 different species.
The dataset can be download from Olympus.
The data file names are:
- images.npy
- Labels.csv
Due to the large volume of data, the images were converted to the images.npy file and the labels are also put into Labels.csv, so that you can work on the data/project seamlessly without having to worry about the high data volume.
The goal of the project is to create a classifier capable of determining a plant's species from an image.
List of Species
- Black-grass
- Charlock
- Cleavers
- Common Chickweed
- Common Wheat
- Fat Hen
- Loose Silky-bent
- Maize
- Scentless Mayweed
- Shepherds Purse
- Small-flowered Cranesbill
- Sugar beet
Development Environment¶
Local development with Visual Studio Code.
Jupyter Notebook and Python 3.11.7 with Anaconda3.
Google Colab/Drive not used.
Generated HTML using the jupyter cli
jupyter nbconvert --to html RobBarker_CV_PlantSeeding_FC.html.ipynbAdded --- (markdown) lines for easier readability for myself.
Formatting Notes¶
- Moved helper functions into separate sections according to task.
- Added line separators for readability.
Importing necessary libraries¶
In [232]:
# Installing the libraries with the specified version.
# uncomment and run the following line if Google Colab is being used
%pip install tensorflow==2.15.0 scikit-learn==1.2.2 seaborn==0.13.1 matplotlib==3.7.1 numpy==1.25.2 pandas==1.5.3 opencv-python==4.8.0.76 -q --user
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import pandas as pd
import math
import os
import cv2 # Importing openCV for image processing
# Tensorflow modules
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator # Importing the ImageDataGenerator for data augmentation
from tensorflow.keras.models import Sequential # Importing the sequential module to define a sequential model
from tensorflow.keras.layers import Input,Dense,Dropout,Flatten,Conv2D,MaxPooling2D,BatchNormalization # Defining all the layers to build our CNN Model
from tensorflow.keras.optimizers import Adam,SGD
from tensorflow.keras.callbacks import LearningRateScheduler, EarlyStopping
from tensorflow.keras.applications import MobileNet
from tensorflow.keras.models import Model
from tensorflow.keras.applications import VGG16
# Importing the optimizers which can be used in our model
from sklearn import preprocessing # Importing the preprocessing module to preprocess the data
from sklearn.model_selection import train_test_split # Importing train_test_split function to split the data into train and test
from sklearn.metrics import confusion_matrix # Importing confusion_matrix to plot the confusion matrix
from sklearn.preprocessing import LabelBinarizer
from sklearn import metrics
# Display images using OpenCV
#from google.colab.patches import cv2_imshow # Importing cv2_imshow from google.patches to display images
from sklearn.model_selection import train_test_split
from tensorflow.keras import backend
from keras.callbacks import ReduceLROnPlateau
import random
from sklearn.metrics import f1_score, precision_score, recall_score, accuracy_score
# To suppress warnings.
import warnings
warnings.filterwarnings("ignore", category=FutureWarning)
warnings.filterwarnings("ignore", category=DeprecationWarning)
warnings.filterwarnings("ignore", category=UserWarning)
Note: you may need to restart the kernel to use updated packages.
Loading the dataset¶
In [87]:
# Load dataset.
plant_df_org = pd.read_csv("/Users/barkz/Desktop/GL Projects/Plant-Seeding-Classification/Labels.csv")
plant_df = plant_df_org.copy()
# Load the image file of dataset
plant_images = np.load("/Users/barkz/Desktop/GL Projects/Plant-Seeding-Classification/images.npy")
Summarization of all the data set elements that include:, Top 5 rows, bottom 5 rows, data info/type, data shape, image information (size, pixels, color) and distribution.
In [3]:
# Check the first few rows of the dataset.
plant_df.head()
Out[3]:
| Label | |
|---|---|
| 0 | Small-flowered Cranesbill |
| 1 | Small-flowered Cranesbill |
| 2 | Small-flowered Cranesbill |
| 3 | Small-flowered Cranesbill |
| 4 | Small-flowered Cranesbill |
In [4]:
# Retrieve last few rows of the copied dataset.
plant_df.tail()
Out[4]:
| Label | |
|---|---|
| 4745 | Loose Silky-bent |
| 4746 | Loose Silky-bent |
| 4747 | Loose Silky-bent |
| 4748 | Loose Silky-bent |
| 4749 | Loose Silky-bent |
In [5]:
# Get dataset information.
plant_df.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 4750 entries, 0 to 4749 Data columns (total 1 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 Label 4750 non-null object dtypes: object(1) memory usage: 37.2+ KB
Observations:
- Dataset contains all 4,750 object datatypes.
In [6]:
# Check for missing values in the dataset, Null.
null_values = plant_df.isnull().sum()
# Check for missing values in the dataset, NaN.
nan_values = plant_df.isna().sum()
# Output if there are any missing data points in the dataset.
if null_values.sum() or nan_values.sum() > 0:
print("There are missing data points in the dataset.")
# List columns with missing values.
null_columns = null_values[null_values > 0]
nan_columns = nan_values[nan_values > 0]
print("Columns with null values:")
print(null_columns)
print("Columns with NaN values:")
print(nan_columns)
else:
print("There are no Nan or null data points in the dataset.")
There are no Nan or null data points in the dataset.
Observations
- There are no NaN or null values in the dataset.
In [7]:
# View plant images dataset.
plant_images
Out[7]:
array([[[[ 35, 52, 78],
[ 36, 49, 76],
[ 31, 45, 69],
...,
[ 78, 95, 114],
[ 76, 93, 110],
[ 80, 95, 109]],
[[ 33, 46, 68],
[ 37, 50, 73],
[ 48, 65, 83],
...,
[ 81, 96, 113],
[ 74, 89, 105],
[ 83, 95, 109]],
[[ 34, 50, 68],
[ 35, 52, 72],
[ 70, 85, 101],
...,
[ 83, 97, 112],
[ 79, 94, 108],
[ 79, 94, 107]],
...,
[[ 35, 50, 69],
[ 42, 57, 73],
[ 42, 57, 72],
...,
[ 60, 76, 92],
[ 67, 81, 97],
[ 64, 77, 95]],
[[ 36, 52, 67],
[ 48, 63, 78],
[ 41, 57, 73],
...,
[ 44, 66, 83],
[ 58, 76, 91],
[ 57, 74, 90]],
[[ 44, 58, 70],
[ 43, 57, 73],
[ 40, 55, 72],
...,
[ 41, 70, 92],
[ 55, 78, 97],
[ 61, 79, 96]]],
[[[ 30, 47, 63],
[ 30, 50, 60],
[ 34, 47, 63],
...,
[ 48, 59, 74],
[ 42, 54, 69],
[ 44, 56, 70]],
[[ 30, 49, 67],
[ 26, 47, 60],
[ 30, 40, 61],
...,
[ 50, 64, 76],
[ 52, 67, 78],
[ 45, 56, 72]],
[[ 23, 46, 65],
[ 27, 48, 64],
[ 25, 40, 59],
...,
[ 39, 59, 81],
[ 47, 62, 79],
[ 42, 54, 69]],
...,
[[ 32, 54, 72],
[ 58, 82, 95],
[ 72, 96, 109],
...,
[ 60, 80, 99],
[ 50, 72, 92],
[ 45, 64, 84]],
[[ 31, 51, 67],
[ 25, 50, 64],
[ 38, 64, 80],
...,
[ 63, 83, 101],
[ 57, 78, 96],
[ 50, 69, 89]],
[[ 18, 32, 56],
[ 16, 27, 50],
[ 34, 49, 71],
...,
[ 59, 84, 101],
[ 55, 80, 97],
[ 39, 59, 82]]],
[[[154, 149, 144],
[162, 156, 152],
[161, 154, 151],
...,
[175, 180, 183],
[177, 181, 184],
[175, 179, 179]],
[[154, 150, 147],
[162, 156, 154],
[154, 147, 145],
...,
[168, 173, 180],
[171, 175, 181],
[173, 176, 180]],
[[151, 149, 149],
[159, 155, 156],
[158, 154, 152],
...,
[173, 176, 180],
[178, 182, 186],
[179, 182, 185]],
...,
[[ 58, 72, 96],
[ 69, 83, 105],
[ 69, 84, 104],
...,
[175, 170, 167],
[176, 169, 167],
[176, 167, 165]],
[[ 64, 76, 98],
[ 77, 87, 108],
[ 79, 91, 109],
...,
[177, 172, 170],
[179, 173, 171],
[176, 169, 166]],
[[ 68, 80, 99],
[ 78, 89, 108],
[ 79, 92, 108],
...,
[179, 173, 170],
[177, 171, 166],
[178, 171, 167]]],
...,
[[[163, 165, 170],
[ 55, 56, 67],
[ 58, 56, 60],
...,
[ 46, 47, 58],
[ 49, 51, 60],
[ 53, 54, 64]],
[[101, 94, 120],
[ 60, 58, 68],
[ 57, 58, 62],
...,
[ 47, 48, 58],
[ 55, 57, 65],
[ 50, 54, 64]],
[[ 64, 56, 76],
[ 64, 60, 67],
[ 54, 50, 55],
...,
[ 47, 50, 60],
[ 48, 51, 62],
[ 45, 49, 60]],
...,
[[ 30, 43, 64],
[ 43, 60, 86],
[ 82, 102, 125],
...,
[ 42, 58, 76],
[ 44, 59, 79],
[ 48, 64, 81]],
[[ 36, 47, 62],
[ 27, 41, 64],
[ 42, 59, 89],
...,
[ 42, 60, 76],
[ 41, 59, 77],
[ 42, 60, 76]],
[[ 30, 41, 57],
[ 26, 40, 59],
[ 30, 45, 68],
...,
[ 41, 59, 74],
[ 39, 60, 73],
[ 40, 57, 74]]],
[[[116, 142, 153],
[107, 137, 152],
[110, 141, 154],
...,
[ 50, 74, 92],
[ 55, 78, 96],
[ 49, 70, 91]],
[[101, 128, 138],
[ 95, 128, 145],
[115, 145, 154],
...,
[ 46, 69, 86],
[ 47, 70, 91],
[ 48, 71, 90]],
[[ 47, 61, 78],
[ 77, 100, 117],
[101, 132, 140],
...,
[ 25, 36, 67],
[ 40, 61, 87],
[ 44, 71, 89]],
...,
[[ 42, 58, 79],
[ 44, 57, 77],
[ 39, 54, 72],
...,
[ 54, 71, 93],
[ 54, 74, 90],
[ 41, 56, 73]],
[[ 38, 63, 92],
[ 34, 60, 87],
[ 40, 62, 81],
...,
[ 59, 75, 94],
[ 50, 70, 88],
[ 40, 57, 73]],
[[ 33, 61, 93],
[ 33, 62, 95],
[ 42, 70, 94],
...,
[ 52, 67, 89],
[ 49, 67, 87],
[ 34, 51, 71]]],
[[[ 53, 66, 84],
[ 45, 56, 74],
[ 52, 67, 86],
...,
[ 50, 71, 95],
[ 83, 106, 124],
[ 65, 86, 113]],
[[ 54, 68, 87],
[ 38, 48, 63],
[ 29, 44, 67],
...,
[ 63, 85, 105],
[ 64, 89, 112],
[ 71, 95, 115]],
[[ 49, 63, 80],
[ 60, 80, 104],
[ 78, 107, 127],
...,
[ 41, 70, 73],
[ 42, 77, 80],
[ 57, 83, 95]],
...,
[[ 50, 65, 79],
[ 50, 65, 84],
[ 55, 74, 94],
...,
[ 28, 52, 79],
[ 31, 50, 74],
[ 32, 51, 72]],
[[ 59, 75, 89],
[ 57, 74, 91],
[ 57, 79, 96],
...,
[ 34, 56, 79],
[ 71, 90, 102],
[ 72, 89, 104]],
[[ 68, 85, 103],
[ 66, 80, 96],
[ 67, 86, 100],
...,
[ 70, 87, 101],
[ 68, 82, 95],
[ 68, 84, 101]]]], dtype=uint8)
In [8]:
# Retrieve number of rows and columns using the shape attribute of the DataFrame.
rows, columns = plant_df.shape
# Print the number of rows and columns from the dataset. Output is formatted into thousands.
# There are only 9 columns but for consistency, the output includes (,) format.
print(f'Number of Rows: {rows:,}')
print(f'Number of Columns: {columns:,}')
Number of Rows: 4,750 Number of Columns: 1
In [9]:
# Retrieve number of rows and columns using the shape attribute of the images dataset.
plant_images.shape
Out[9]:
(4750, 128, 128, 3)
Observations
- Plant Labels
- There are 4,750 rows and 1 columns in the dataset.
- Plant Images (4750, 128, 128, 3)
- 4750: This is the number of images in the dataset.
- 128: This is the height of each image has a height of 128 pixels.
- 128: This is the width of each image in pixels, a width of 128 pixels.
- 3: This represents the number of color channels in each image. Since the value is 3, it indicates that the images are in RGB format, with three channels corresponding to Red, Green, and Blue.
In [10]:
# Check datattypes to ensure pixels within the expected range.
print(f'Data type of plant_images: {plant_images.dtype}')
print(f'Min pixel value: {plant_images.min()}')
print(f'Max pixel value: {plant_images.max()}')
Data type of plant_images: uint8 Min pixel value: 0 Max pixel value: 255
Observations
- Pixel values are within range of 0 - 255.
In [11]:
# Check different plants in the dataset.
class_distribution = plant_df['Label'].value_counts()
print(class_distribution)
Loose Silky-bent 654 Common Chickweed 611 Scentless Mayweed 516 Small-flowered Cranesbill 496 Fat Hen 475 Charlock 390 Sugar beet 385 Cleavers 287 Black-grass 263 Shepherds Purse 231 Common wheat 221 Maize 221 Name: Label, dtype: int64
Observations
- There are 12 uniques plant types in the dataset.
In [12]:
# Check for any NaN values in the array.
print(f'Any NaN values in plant_images: {np.isnan(plant_images).any()}')
Any NaN values in plant_images: False
Observations
- There are no NaN values in the dataset.
In [13]:
# Verify that all images have consistent dimensions.
unique_shapes = np.unique([img.shape for img in plant_images], axis=0)
print(f'Unique image shapes: {unique_shapes}')
Unique image shapes: [[128 128 3]]
Observations
- All images have consistent, 128x128 dimensions with 3 channels (RGB).
In [14]:
# Get statistical information (mean, standard deviation) for pixel values.
print(f'Mean pixel value: {plant_images.mean()}')
print(f'Standard deviation of pixel values: {plant_images.std()}')
Mean pixel value: 70.04363745545504 Standard deviation of pixel values: 31.996876308515
Observations
- Mean pixel value: 70
- STD of pixel values: 32
- EDA is an important part of any project involving data.
- It is important to investigate and understand the data better before building a model with it.
- A few questions have been mentioned below which will help you understand the data better.
- A thorough analysis of the data, in addition to the questions mentioned below, should be done.
- How are these different category plant images different from each other?
- Is the dataset provided an imbalance? (Check with using bar plots)
Helper Functions¶
In [88]:
# Plot random images from each of the class.
def plot_images(images,labels):
num_classes=10 # Number of Classes
categories=np.unique(labels)
keys=dict(labels['Label']) # Obtaing the unique classes from y_train
rows = 3 # Defining number of rows=3
cols = 4 # Defining number of columns=4
fig = plt.figure(figsize=(10, 8)) # Defining the figure size to 10x8
for i in range(cols):
for j in range(rows):
random_index = np.random.randint(0, len(labels)) # Generating random indices from the data and plotting the images
ax = fig.add_subplot(rows, cols, i * rows + j + 1) # Adding subplots with 3 rows and 4 columns
ax.imshow(images[random_index, :])# Plotting the image
ax.set_title(keys[random_index])
plt.show()
Analysis¶
In [89]:
# Plot the 12 unique plant images.plant_images.
plot_images(plant_images,plant_df)
In [90]:
# Count each label and sort them in descending order.
label_counts = plant_df['Label'].value_counts().sort_values(ascending=False)
# Create a count plot with the sorted labels.
ax = sns.countplot(x=plant_df['Label'], order=label_counts.index)
plt.xticks(rotation='vertical')
# Add values to the bars.
for p in ax.patches:
ax.annotate(f'{int(p.get_height())}', (p.get_x() + p.get_width() / 2., p.get_height()),
ha='center', va='center', xytext=(0, 10), textcoords='offset points')
plt.show()
In [91]:
skewness = plant_df['Label'].value_counts().skew()
print(f'Skewness of the Label data: {skewness}')
Skewness of the Label data: 0.34509359058206984
Observations
- Loose Silky-bent is the most recurring plant at 654.
- Common Chickweed is second with 611.
- The data is slightly right skewed.
Convert the BGR images to RGB images.¶
In [92]:
# Converting the images from BGR to RGB using cvtColor function of OpenCV.
for i in range(len(plant_images)):
plant_images[i] = cv2.cvtColor(plant_images[i], cv2.COLOR_BGR2RGB)
# Check if an image is in RGB format.
def is_rgb(image):
# Are the first pixel blue and red channels are swapped.
return (image[..., 0] != image[..., 2]).any()
# Verify the conversion for each image.
conversion_successful = all(is_rgb(image) for image in plant_images)
# Output success or not.
if conversion_successful:
print("All images have been successfully converted to RGB.")
else:
print("Some images were not converted to RGB.")
All images have been successfully converted to RGB.
Observations
- All images have been converted properly.
Resize the images¶
In [93]:
# Resize images 128x128 to 64x64.
plant_images_resized=[]
height = 64
width = 64
dimensions = (width, height)
for i in range(len(plant_images)):
plant_images_resized.append(
cv2.resize(
plant_images[i],
dimensions,
interpolation=cv2.INTER_LINEAR
)
)
# Check if an image has the correct dimensions.
def is_correct_size(image, target_dimensions):
return image.shape[:2] == target_dimensions
# Verify the resizing for each image.
target_dimensions = (height, width)
resizing_successful = all(is_correct_size(image, target_dimensions) for image in plant_images_resized)
# Output success or not.
if resizing_successful:
print("All images have been successfully resized to 64x64.")
else:
print("Some images were not resized correctly.")
All images have been successfully resized to 64x64.
Images pre Resize¶
In [94]:
# Display the images before resizing.
# Notice the difference in the image quality.
unique_labels = plant_df['Label'].unique()
# Plot one image for each unique label.
plt.figure(figsize=(15, 15))
num_rows = 4
num_cols = 3
max_images = num_rows * num_cols
for i, label in enumerate(unique_labels[:max_images]):
# Find the first image corresponding to the current label.
image_index = plant_df[plant_df['Label'] == label].index[0]
image = plant_images[image_index]
# Plot the image.
plt.subplot(num_rows, num_cols, i + 1)
plt.imshow(image)
plt.title(label)
plt.axis('off')
plt.tight_layout()
plt.show()
Observations
- Images look good and clear.
In [95]:
# Plot one select image from original images to show individual quality.
plt.imshow(plant_images[4])
Out[95]:
<matplotlib.image.AxesImage at 0x36889e490>
Images post Resize¶
In [96]:
# Display the images after resizing.
# Notice the difference in the image quality.
unique_labels = plant_df['Label'].unique()
# Plot one image for each unique label
plt.figure(figsize=(15, 15))
num_rows = 4
num_cols = 3
max_images = num_rows * num_cols
for i, label in enumerate(unique_labels[:max_images]):
# Find the first image corresponding to the current label.
image_index = plant_df[plant_df['Label'] == label].index[0]
image = plant_images_resized[image_index]
# Plot the image
plt.subplot(num_rows, num_cols, i + 1)
plt.imshow(image)
plt.title(label)
plt.axis('off')
plt.tight_layout()
plt.show()
Observations
- Images a bit more pixelated.
In [97]:
# Plot one image from resized images to show individual quality.
plt.imshow(plant_images_resized[4])
Out[97]:
<matplotlib.image.AxesImage at 0x36619e310>
Observations
- Looking individually at a plant it is pixelated.
Data Preparation for Modeling¶
- Before you proceed to build a model, you need to split the data into train, test, and validation to be able to evaluate the model that you build on the train data
- You'll have to encode categorical features and scale the pixel values.
- You will build a model using the train data and then check its performance
Split the dataset¶
In [98]:
# Print the number of resized images.
print(f"Length of images_decreased: {len(plant_images_resized)}")
# Print the number of labels in the DataFrame.
print(f"Length of labels: {len(plant_df)}")
labels_array = plant_df['Label'].to_numpy()
# Print the shape of the resized images array.
print(f"Shape of images_decreased: {np.shape(plant_images_resized)}")
# Ensure the number of images matches.
images_decreased = plant_images_resized[:len(labels_array)]
# Split the data.
X_temp, X_test, y_temp, y_test = train_test_split(images_decreased, labels_array, test_size=0.1, random_state=42, stratify=labels_array)
# Further split the data into training and validation sets.
X_train, X_val, y_train, y_val = train_test_split(X_temp, y_temp, test_size=0.1, random_state=42, stratify=y_temp)
Length of images_decreased: 4750 Length of labels: 4750 Shape of images_decreased: (4750, 64, 64, 3)
In [99]:
# Convert the lists to numpy arrays.
X_train = np.array(X_train)
X_val = np.array(X_val)
X_test = np.array(X_test)
# Print shape of the training, validation, and test sets.
print(X_train.shape, y_train.shape)
print(X_val.shape, y_val.shape)
print(X_test.shape, y_test.shape)
# Calculate the total number of samples.
total_samples = len(X_train) + len(X_val) + len(X_test)
# Calculate the percentage of samples in each set.
train_percentage = (len(X_train) / total_samples) * 100
val_percentage = (len(X_val) / total_samples) * 100
test_percentage = (len(X_test) / total_samples) * 100
# Data split percentages.
print(f"\nTraining set: {train_percentage:.2f}%")
print(f"Validation set: {val_percentage:.2f}%")
print(f"Test set: {test_percentage:.2f}%")
(3847, 64, 64, 3) (3847,) (428, 64, 64, 3) (428,) (475, 64, 64, 3) (475,) Training set: 80.99% Validation set: 9.01% Test set: 10.00%
Encode the target labels¶
In [100]:
# Convert labels from names to one hot vectors.
enc = LabelBinarizer() # Complete the code to intialize the labelBinarizer
y_train_encoded = enc.fit_transform(y_train) # Complete the code to fit and transform y_train
y_val_encoded=enc.transform(y_val) # Complete the code to transform y_val
y_test_encoded=enc.transform(y_test)
In [101]:
y_train_encoded.shape,y_val_encoded.shape,y_test_encoded.shape # Complete the code to check the shape of train, validation and test data
Out[101]:
((3847, 12), (428, 12), (475, 12))
Data Normalization¶
In [102]:
# Complete the code to normalize the image pixels of train, test and validation data.
X_train_normalized = X_train.astype('float32')/255.0
X_val_normalized = X_val.astype('float32')/255.0
X_test_normalized = X_test.astype('float32')/255.0
Helper Functions¶
In [173]:
# This function will return a dataframe of metrics.
def model_performance_classification_sklearn(model, predictors, target, name):
"""
Function to compute different metrics to check classification model performance
model: classifier
predictors: independent variables
target: dependent variable
"""
# predicting using the independent variables
pred_p = model.predict(predictors)
pred = np.argmax(pred_p,axis=1)
acc = accuracy_score(target, pred) # to compute Accuracy
recall = recall_score(target, pred, average='weighted') # to compute Recall
precision = precision_score(target, pred, average='weighted') # to compute Precision
f1 = f1_score(target, pred, average='weighted') # to compute F1-score
# creating a dataframe of metrics
df_perf = pd.DataFrame(
{"Model": name, "Accuracy": acc, "Recall": recall, "Precision": precision, "F1": f1,},
index=[0],
)
return df_perf
Building Model 1¶
In [174]:
# Clearing backend.
backend.clear_session()
In [175]:
# Fixing the seed for random number generators.
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
In [176]:
# Intializing a sequential model
model1 = Sequential() # Complete the code to intialize a sequential model
# Assuming you have 10 classes in your classification problem
num_classes = 12
# Complete the code to add the first conv layer with 128 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension of images
model1.add(Conv2D(32, (3, 3), activation='relu', padding="same", input_shape=(64, 64, 3)))
# Complete the code to add the max pooling to reduce the size of output of first conv layer
model1.add(MaxPooling2D((2, 2), padding = 'same'))
# Complete the code to create two similar convolution and max-pooling layers activation = relu
model1.add(Conv2D(64, (3, 3), activation='relu', padding="same"))
model1.add(MaxPooling2D((2, 2), padding = 'same'))
model1.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model1.add(MaxPooling2D((2, 2), padding = 'same'))
# Complete the code to flatten the output of the conv layer after max pooling to make it ready for creating dense connections
model1.add(Flatten())
# Complete the code to add a fully connected dense layer with 16 neurons
model1.add(Dense(128, activation='relu'))
model1.add(Dropout(0.3))
# Complete the code to add the output layer with 12 neurons and activation functions as softmax since this is a multi-class classification problem
model1.add(Dense(num_classes, activation='softmax'))
# Complete the code to use the Adam Optimizer
opt=Adam()
# Complete the code to Compile the model using suitable metric for loss fucntion
model1.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
# Complete the code to generate the summary of the model
model1.summary()
WARNING:absl:At this time, the v2.11+ optimizer `tf.keras.optimizers.Adam` runs slowly on M1/M2 Macs, please use the legacy Keras optimizer instead, located at `tf.keras.optimizers.legacy.Adam`.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 64, 64, 32) 896
max_pooling2d (MaxPooling2 (None, 32, 32, 32) 0
D)
conv2d_1 (Conv2D) (None, 32, 32, 64) 18496
max_pooling2d_1 (MaxPoolin (None, 16, 16, 64) 0
g2D)
conv2d_2 (Conv2D) (None, 16, 16, 32) 18464
max_pooling2d_2 (MaxPoolin (None, 8, 8, 32) 0
g2D)
flatten (Flatten) (None, 2048) 0
dense (Dense) (None, 128) 262272
dropout (Dropout) (None, 128) 0
dense_1 (Dense) (None, 12) 1548
=================================================================
Total params: 301676 (1.15 MB)
Trainable params: 301676 (1.15 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
In [177]:
# Fit the model on train and validation data.
history_1 = model1.fit(
X_train_normalized, y_train_encoded,
epochs=50,
validation_data=(X_val_normalized, y_val_encoded),
batch_size=64,
verbose=2
)
# Convert one-hot encoded labels back to original class labels.
y_test_original = np.argmax(y_test_encoded, axis=1)
# Report the performance of the model.
model_performance_classification_sklearn(model1, X_test_normalized, y_test_original, 'Model 1')
Epoch 1/50 61/61 - 5s - loss: 2.4023 - accuracy: 0.1648 - val_loss: 2.2613 - val_accuracy: 0.2780 - 5s/epoch - 75ms/step Epoch 2/50 61/61 - 4s - loss: 1.9272 - accuracy: 0.3379 - val_loss: 1.6082 - val_accuracy: 0.4206 - 4s/epoch - 69ms/step Epoch 3/50 61/61 - 4s - loss: 1.6417 - accuracy: 0.4159 - val_loss: 1.4895 - val_accuracy: 0.4743 - 4s/epoch - 68ms/step Epoch 4/50 61/61 - 4s - loss: 1.4782 - accuracy: 0.4669 - val_loss: 1.3437 - val_accuracy: 0.5164 - 4s/epoch - 73ms/step Epoch 5/50 61/61 - 4s - loss: 1.3565 - accuracy: 0.5134 - val_loss: 1.2050 - val_accuracy: 0.5584 - 4s/epoch - 73ms/step Epoch 6/50 61/61 - 4s - loss: 1.1858 - accuracy: 0.5792 - val_loss: 1.1135 - val_accuracy: 0.6262 - 4s/epoch - 73ms/step Epoch 7/50 61/61 - 5s - loss: 1.0857 - accuracy: 0.6223 - val_loss: 0.9877 - val_accuracy: 0.6565 - 5s/epoch - 75ms/step Epoch 8/50 61/61 - 5s - loss: 0.9864 - accuracy: 0.6512 - val_loss: 0.9646 - val_accuracy: 0.6589 - 5s/epoch - 82ms/step Epoch 9/50 61/61 - 4s - loss: 0.9326 - accuracy: 0.6629 - val_loss: 0.9491 - val_accuracy: 0.6893 - 4s/epoch - 71ms/step Epoch 10/50 61/61 - 4s - loss: 0.8617 - accuracy: 0.6925 - val_loss: 0.9647 - val_accuracy: 0.6495 - 4s/epoch - 69ms/step Epoch 11/50 61/61 - 4s - loss: 0.8188 - accuracy: 0.7073 - val_loss: 0.8319 - val_accuracy: 0.7360 - 4s/epoch - 71ms/step Epoch 12/50 61/61 - 5s - loss: 0.7462 - accuracy: 0.7341 - val_loss: 0.8071 - val_accuracy: 0.7336 - 5s/epoch - 77ms/step Epoch 13/50 61/61 - 5s - loss: 0.7141 - accuracy: 0.7468 - val_loss: 0.8656 - val_accuracy: 0.6963 - 5s/epoch - 77ms/step Epoch 14/50 61/61 - 4s - loss: 0.7063 - accuracy: 0.7528 - val_loss: 0.8023 - val_accuracy: 0.7360 - 4s/epoch - 72ms/step Epoch 15/50 61/61 - 4s - loss: 0.6518 - accuracy: 0.7549 - val_loss: 0.7546 - val_accuracy: 0.7780 - 4s/epoch - 72ms/step Epoch 16/50 61/61 - 5s - loss: 0.6109 - accuracy: 0.7887 - val_loss: 0.7816 - val_accuracy: 0.7617 - 5s/epoch - 77ms/step Epoch 17/50 61/61 - 4s - loss: 0.6025 - accuracy: 0.7894 - val_loss: 0.8032 - val_accuracy: 0.7687 - 4s/epoch - 70ms/step Epoch 18/50 61/61 - 4s - loss: 0.5621 - accuracy: 0.8043 - val_loss: 0.7405 - val_accuracy: 0.7757 - 4s/epoch - 71ms/step Epoch 19/50 61/61 - 4s - loss: 0.5372 - accuracy: 0.8126 - val_loss: 0.7659 - val_accuracy: 0.7640 - 4s/epoch - 69ms/step Epoch 20/50 61/61 - 4s - loss: 0.5034 - accuracy: 0.8199 - val_loss: 0.7974 - val_accuracy: 0.7570 - 4s/epoch - 67ms/step Epoch 21/50 61/61 - 4s - loss: 0.4758 - accuracy: 0.8308 - val_loss: 0.7778 - val_accuracy: 0.7827 - 4s/epoch - 66ms/step Epoch 22/50 61/61 - 4s - loss: 0.4656 - accuracy: 0.8310 - val_loss: 0.8227 - val_accuracy: 0.7664 - 4s/epoch - 68ms/step Epoch 23/50 61/61 - 4s - loss: 0.4296 - accuracy: 0.8456 - val_loss: 0.7653 - val_accuracy: 0.7921 - 4s/epoch - 71ms/step Epoch 24/50 61/61 - 4s - loss: 0.4300 - accuracy: 0.8435 - val_loss: 0.8050 - val_accuracy: 0.7874 - 4s/epoch - 68ms/step Epoch 25/50 61/61 - 4s - loss: 0.4561 - accuracy: 0.8375 - val_loss: 0.7166 - val_accuracy: 0.7991 - 4s/epoch - 68ms/step Epoch 26/50 61/61 - 4s - loss: 0.3787 - accuracy: 0.8534 - val_loss: 0.8136 - val_accuracy: 0.7850 - 4s/epoch - 71ms/step Epoch 27/50 61/61 - 4s - loss: 0.3850 - accuracy: 0.8557 - val_loss: 0.8036 - val_accuracy: 0.7921 - 4s/epoch - 69ms/step Epoch 28/50 61/61 - 4s - loss: 0.3643 - accuracy: 0.8682 - val_loss: 0.7820 - val_accuracy: 0.7804 - 4s/epoch - 70ms/step Epoch 29/50 61/61 - 4s - loss: 0.3415 - accuracy: 0.8752 - val_loss: 0.8361 - val_accuracy: 0.7804 - 4s/epoch - 68ms/step Epoch 30/50 61/61 - 4s - loss: 0.3511 - accuracy: 0.8703 - val_loss: 0.7866 - val_accuracy: 0.8014 - 4s/epoch - 70ms/step Epoch 31/50 61/61 - 4s - loss: 0.3419 - accuracy: 0.8716 - val_loss: 0.7989 - val_accuracy: 0.7874 - 4s/epoch - 68ms/step Epoch 32/50 61/61 - 4s - loss: 0.3100 - accuracy: 0.8859 - val_loss: 0.8450 - val_accuracy: 0.7804 - 4s/epoch - 68ms/step Epoch 33/50 61/61 - 4s - loss: 0.2979 - accuracy: 0.8906 - val_loss: 0.8013 - val_accuracy: 0.7897 - 4s/epoch - 69ms/step Epoch 34/50 61/61 - 4s - loss: 0.2769 - accuracy: 0.8976 - val_loss: 0.8942 - val_accuracy: 0.7991 - 4s/epoch - 71ms/step Epoch 35/50 61/61 - 4s - loss: 0.2977 - accuracy: 0.8895 - val_loss: 0.9296 - val_accuracy: 0.7827 - 4s/epoch - 69ms/step Epoch 36/50 61/61 - 4s - loss: 0.2707 - accuracy: 0.9002 - val_loss: 0.7757 - val_accuracy: 0.8178 - 4s/epoch - 71ms/step Epoch 37/50 61/61 - 5s - loss: 0.2665 - accuracy: 0.9023 - val_loss: 0.8405 - val_accuracy: 0.7827 - 5s/epoch - 75ms/step Epoch 38/50 61/61 - 5s - loss: 0.2609 - accuracy: 0.9030 - val_loss: 0.8537 - val_accuracy: 0.8084 - 5s/epoch - 90ms/step Epoch 39/50 61/61 - 5s - loss: 0.2623 - accuracy: 0.9043 - val_loss: 0.9004 - val_accuracy: 0.7944 - 5s/epoch - 76ms/step Epoch 40/50 61/61 - 5s - loss: 0.2652 - accuracy: 0.8989 - val_loss: 0.8888 - val_accuracy: 0.8014 - 5s/epoch - 79ms/step Epoch 41/50 61/61 - 5s - loss: 0.2159 - accuracy: 0.9223 - val_loss: 1.0460 - val_accuracy: 0.7827 - 5s/epoch - 89ms/step Epoch 42/50 61/61 - 5s - loss: 0.2349 - accuracy: 0.9132 - val_loss: 0.9536 - val_accuracy: 0.8037 - 5s/epoch - 89ms/step Epoch 43/50 61/61 - 5s - loss: 0.2075 - accuracy: 0.9171 - val_loss: 0.9603 - val_accuracy: 0.8061 - 5s/epoch - 87ms/step Epoch 44/50 61/61 - 5s - loss: 0.2110 - accuracy: 0.9246 - val_loss: 0.8664 - val_accuracy: 0.7991 - 5s/epoch - 86ms/step Epoch 45/50 61/61 - 6s - loss: 0.2111 - accuracy: 0.9184 - val_loss: 0.9856 - val_accuracy: 0.7967 - 6s/epoch - 91ms/step Epoch 46/50 61/61 - 6s - loss: 0.1916 - accuracy: 0.9314 - val_loss: 0.8779 - val_accuracy: 0.8294 - 6s/epoch - 103ms/step Epoch 47/50 61/61 - 7s - loss: 0.2469 - accuracy: 0.9082 - val_loss: 0.9858 - val_accuracy: 0.7804 - 7s/epoch - 117ms/step Epoch 48/50 61/61 - 7s - loss: 0.1910 - accuracy: 0.9275 - val_loss: 0.8978 - val_accuracy: 0.8061 - 7s/epoch - 109ms/step Epoch 49/50 61/61 - 6s - loss: 0.1574 - accuracy: 0.9420 - val_loss: 0.9307 - val_accuracy: 0.8224 - 6s/epoch - 104ms/step Epoch 50/50 61/61 - 6s - loss: 0.1839 - accuracy: 0.9322 - val_loss: 1.0290 - val_accuracy: 0.8014 - 6s/epoch - 105ms/step 15/15 [==============================] - 0s 14ms/step
Out[177]:
| Model | Accuracy | Recall | Precision | F1 | |
|---|---|---|---|---|---|
| 0 | Model 1 | 0.776842 | 0.776842 | 0.779542 | 0.774651 |
In [178]:
# Plot the loss and accuracy of the model.
plt.plot(history_1.history['accuracy'])
plt.plot(history_1.history['val_accuracy'])
plt.title('Model 1 Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations:
- The training accuracy is significantly higher (93.22%) compared to the validation accuracy (80.14%). This shows the model is performing much better on the training data than on the validation data.
- The validation loss decreases initially but starts to increase towards the end of training.
- Model is overfitting. The high training accuracy combined with a significantly lower validation accuracy and increasing validation loss towards the end of training are strong indicators of overfitting.
In [179]:
# Evaluate the model on the test data.
accuracy = model1.evaluate(X_test_normalized, y_test_encoded, verbose=2)
# Print the accuracy of the model on test data.
accuracy
15/15 - 0s - loss: 0.9387 - accuracy: 0.7768 - 238ms/epoch - 16ms/step
Out[179]:
[0.9386667609214783, 0.7768421173095703]
Observations:
- The model achieved a loss of 0.9387 and an accuracy of 77.68% on the test data.
- The accuracy indicates that the model performs reasonably well, correctly predicting about 78% of the test samples. There is room for improvement, as indicated by the loss value.
In [180]:
# Get the output as probablities for each category.
y_pred=model1.predict(X_test_normalized)
15/15 [==============================] - 0s 15ms/step
In [182]:
# Obtaining the categorical values from y_test_encoded and y_pred.
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which
# is also predefined in tensorflow module.
confusion_matrix = tf.math.confusion_matrix(y_test_arg, y_pred_arg) # Complete the code to plot the confusion matrix
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Model 1 Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()
In [183]:
# Plotting the classification report
print("Model 1 Classification Report")
cr=metrics.classification_report(y_test_arg,y_pred_arg) # Complete the code to plot the classification report
print(cr)
Model 1 Classification Report
precision recall f1-score support
0 0.48 0.38 0.43 26
1 0.73 0.95 0.82 39
2 0.79 0.76 0.77 29
3 0.95 0.85 0.90 61
4 0.63 0.77 0.69 22
5 0.81 0.79 0.80 48
6 0.78 0.72 0.75 65
7 0.71 0.77 0.74 22
8 0.71 0.81 0.76 52
9 0.65 0.48 0.55 23
10 0.93 0.86 0.90 50
11 0.82 0.87 0.85 38
accuracy 0.78 475
macro avg 0.75 0.75 0.75 475
weighted avg 0.78 0.78 0.77 475
Obversations
- The model performs well overall with an accuracy of 78%.
In [157]:
# Code to monitor val_accuracy.
learning_rate_reduction = ReduceLROnPlateau(monitor='val_accuracy',
patience=3,
verbose=1,
factor=0.5,
min_lr=0.00001)
Building Model 2¶
Data Augmentation¶
- Rotation
- Fill
In [190]:
# Complete the code to set the rotation_range to 20
train_datagen = ImageDataGenerator(
rotation_range=20,
fill_mode='nearest'
)
In [191]:
# Clear backend session.
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
In [192]:
model2 = Sequential()
# Complete the code to add the first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension images
model2.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(64, 64, 3)))
# Complete the code to add max pooling to reduce the size of output of first conv layer
model2.add(MaxPooling2D((2, 2), padding = 'same'))
model2.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model2.add(MaxPooling2D((2, 2), padding = 'same'))
model2.add(BatchNormalization())
# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model2.add(Flatten())
# Adding a fully connected dense layer with 16 neurons
model2.add(Dense(16, activation='relu'))
# Complete the code to add dropout with dropout_rate=0.3
model2.add(Dropout(0.25))
# Complete the code to add the output layer with 12 neurons and activation functions as softmax since this is a multi-class classification problem
model2.add(Dense(12, activation='softmax'))
# Complete the code to initialize Adam Optimimzer
opt=Adam()
# Complete the code to Compile model
model2.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
# Generating the summary of the model
model2.summary()
WARNING:absl:At this time, the v2.11+ optimizer `tf.keras.optimizers.Adam` runs slowly on M1/M2 Macs, please use the legacy Keras optimizer instead, located at `tf.keras.optimizers.legacy.Adam`.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 64, 64, 64) 1792
max_pooling2d (MaxPooling2 (None, 32, 32, 64) 0
D)
conv2d_1 (Conv2D) (None, 32, 32, 32) 18464
max_pooling2d_1 (MaxPoolin (None, 16, 16, 32) 0
g2D)
batch_normalization (Batch (None, 16, 16, 32) 128
Normalization)
flatten (Flatten) (None, 8192) 0
dense (Dense) (None, 16) 131088
dropout (Dropout) (None, 16) 0
dense_1 (Dense) (None, 12) 204
=================================================================
Total params: 151676 (592.48 KB)
Trainable params: 151612 (592.23 KB)
Non-trainable params: 64 (256.00 Byte)
_________________________________________________________________
In [193]:
# Complete the code to fit the model on train data with batch_size=64 and epochs=30.
epochs = 50
batch_size = 64
history = model2.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
batch_size=batch_size,
shuffle=False),
epochs=epochs,
steps_per_epoch=X_train_normalized.shape[0] // batch_size,
validation_data=(X_val_normalized,y_val_encoded),
verbose=1,callbacks=[learning_rate_reduction])
Epoch 1/50 60/60 [==============================] - 5s 80ms/step - loss: 2.1161 - accuracy: 0.2577 - val_loss: 2.3871 - val_accuracy: 0.1752 - lr: 0.0010 Epoch 2/50 60/60 [==============================] - 5s 79ms/step - loss: 1.6174 - accuracy: 0.4420 - val_loss: 2.2758 - val_accuracy: 0.1822 - lr: 0.0010 Epoch 3/50 60/60 [==============================] - 5s 80ms/step - loss: 1.4076 - accuracy: 0.5096 - val_loss: 2.1772 - val_accuracy: 0.2126 - lr: 0.0010 Epoch 4/50 60/60 [==============================] - 5s 77ms/step - loss: 1.3053 - accuracy: 0.5448 - val_loss: 2.0782 - val_accuracy: 0.3271 - lr: 0.0010 Epoch 5/50 60/60 [==============================] - 5s 76ms/step - loss: 1.2123 - accuracy: 0.5675 - val_loss: 1.7751 - val_accuracy: 0.5607 - lr: 0.0010 Epoch 6/50 60/60 [==============================] - 5s 78ms/step - loss: 1.1298 - accuracy: 0.6019 - val_loss: 1.5988 - val_accuracy: 0.5093 - lr: 0.0010 Epoch 7/50 60/60 [==============================] - 5s 77ms/step - loss: 1.1299 - accuracy: 0.6056 - val_loss: 1.8512 - val_accuracy: 0.3575 - lr: 0.0010 Epoch 8/50 60/60 [==============================] - 5s 78ms/step - loss: 1.0335 - accuracy: 0.6371 - val_loss: 1.3225 - val_accuracy: 0.5631 - lr: 0.0010 Epoch 9/50 60/60 [==============================] - 5s 76ms/step - loss: 1.0372 - accuracy: 0.6278 - val_loss: 1.0833 - val_accuracy: 0.6916 - lr: 0.0010 Epoch 10/50 60/60 [==============================] - 5s 81ms/step - loss: 0.9477 - accuracy: 0.6585 - val_loss: 1.0983 - val_accuracy: 0.6402 - lr: 0.0010 Epoch 11/50 60/60 [==============================] - 5s 77ms/step - loss: 0.9342 - accuracy: 0.6683 - val_loss: 1.5404 - val_accuracy: 0.5397 - lr: 0.0010 Epoch 12/50 60/60 [==============================] - ETA: 0s - loss: 0.8910 - accuracy: 0.6727 Epoch 12: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257. 60/60 [==============================] - 5s 78ms/step - loss: 0.8910 - accuracy: 0.6727 - val_loss: 1.0721 - val_accuracy: 0.6846 - lr: 0.0010 Epoch 13/50 60/60 [==============================] - 5s 80ms/step - loss: 0.8278 - accuracy: 0.7029 - val_loss: 0.7726 - val_accuracy: 0.7500 - lr: 5.0000e-04 Epoch 14/50 60/60 [==============================] - 5s 77ms/step - loss: 0.7863 - accuracy: 0.7119 - val_loss: 0.9914 - val_accuracy: 0.7173 - lr: 5.0000e-04 Epoch 15/50 60/60 [==============================] - 5s 89ms/step - loss: 0.7588 - accuracy: 0.7306 - val_loss: 0.7661 - val_accuracy: 0.7734 - lr: 5.0000e-04 Epoch 16/50 60/60 [==============================] - 5s 83ms/step - loss: 0.7424 - accuracy: 0.7304 - val_loss: 0.7646 - val_accuracy: 0.7570 - lr: 5.0000e-04 Epoch 17/50 60/60 [==============================] - 6s 93ms/step - loss: 0.7429 - accuracy: 0.7301 - val_loss: 0.8963 - val_accuracy: 0.7033 - lr: 5.0000e-04 Epoch 18/50 60/60 [==============================] - 5s 91ms/step - loss: 0.7120 - accuracy: 0.7433 - val_loss: 0.7253 - val_accuracy: 0.7804 - lr: 5.0000e-04 Epoch 19/50 60/60 [==============================] - 5s 81ms/step - loss: 0.6941 - accuracy: 0.7402 - val_loss: 1.0896 - val_accuracy: 0.6729 - lr: 5.0000e-04 Epoch 20/50 60/60 [==============================] - 5s 82ms/step - loss: 0.6779 - accuracy: 0.7497 - val_loss: 0.8503 - val_accuracy: 0.7477 - lr: 5.0000e-04 Epoch 21/50 60/60 [==============================] - ETA: 0s - loss: 0.6655 - accuracy: 0.7563 Epoch 21: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628. 60/60 [==============================] - 5s 81ms/step - loss: 0.6655 - accuracy: 0.7563 - val_loss: 0.7911 - val_accuracy: 0.7780 - lr: 5.0000e-04 Epoch 22/50 60/60 [==============================] - 5s 80ms/step - loss: 0.6380 - accuracy: 0.7637 - val_loss: 0.7252 - val_accuracy: 0.7991 - lr: 2.5000e-04 Epoch 23/50 60/60 [==============================] - 5s 84ms/step - loss: 0.6388 - accuracy: 0.7608 - val_loss: 0.6756 - val_accuracy: 0.8037 - lr: 2.5000e-04 Epoch 24/50 60/60 [==============================] - 5s 83ms/step - loss: 0.6241 - accuracy: 0.7743 - val_loss: 0.6411 - val_accuracy: 0.8224 - lr: 2.5000e-04 Epoch 25/50 60/60 [==============================] - 5s 84ms/step - loss: 0.6142 - accuracy: 0.7698 - val_loss: 0.6364 - val_accuracy: 0.8294 - lr: 2.5000e-04 Epoch 26/50 60/60 [==============================] - 5s 89ms/step - loss: 0.5975 - accuracy: 0.7774 - val_loss: 0.7402 - val_accuracy: 0.7780 - lr: 2.5000e-04 Epoch 27/50 60/60 [==============================] - 5s 89ms/step - loss: 0.6016 - accuracy: 0.7817 - val_loss: 0.7242 - val_accuracy: 0.7897 - lr: 2.5000e-04 Epoch 28/50 60/60 [==============================] - ETA: 0s - loss: 0.5834 - accuracy: 0.7867 Epoch 28: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814. 60/60 [==============================] - 6s 91ms/step - loss: 0.5834 - accuracy: 0.7867 - val_loss: 0.6936 - val_accuracy: 0.8037 - lr: 2.5000e-04 Epoch 29/50 60/60 [==============================] - 6s 99ms/step - loss: 0.5793 - accuracy: 0.7875 - val_loss: 0.6617 - val_accuracy: 0.8294 - lr: 1.2500e-04 Epoch 30/50 60/60 [==============================] - 6s 103ms/step - loss: 0.5583 - accuracy: 0.7988 - val_loss: 0.6533 - val_accuracy: 0.8224 - lr: 1.2500e-04 Epoch 31/50 60/60 [==============================] - ETA: 0s - loss: 0.5738 - accuracy: 0.7941 Epoch 31: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05. 60/60 [==============================] - 7s 112ms/step - loss: 0.5738 - accuracy: 0.7941 - val_loss: 0.6299 - val_accuracy: 0.8154 - lr: 1.2500e-04 Epoch 32/50 60/60 [==============================] - 7s 118ms/step - loss: 0.5555 - accuracy: 0.7965 - val_loss: 0.6240 - val_accuracy: 0.8294 - lr: 6.2500e-05 Epoch 33/50 60/60 [==============================] - 7s 123ms/step - loss: 0.5714 - accuracy: 0.7893 - val_loss: 0.6257 - val_accuracy: 0.8318 - lr: 6.2500e-05 Epoch 34/50 60/60 [==============================] - 8s 124ms/step - loss: 0.5510 - accuracy: 0.7994 - val_loss: 0.6060 - val_accuracy: 0.8364 - lr: 6.2500e-05 Epoch 35/50 60/60 [==============================] - 7s 122ms/step - loss: 0.5659 - accuracy: 0.7983 - val_loss: 0.6220 - val_accuracy: 0.8294 - lr: 6.2500e-05 Epoch 36/50 60/60 [==============================] - 8s 124ms/step - loss: 0.5445 - accuracy: 0.7973 - val_loss: 0.6392 - val_accuracy: 0.8248 - lr: 6.2500e-05 Epoch 37/50 60/60 [==============================] - ETA: 0s - loss: 0.5429 - accuracy: 0.8068 Epoch 37: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05. 60/60 [==============================] - 7s 122ms/step - loss: 0.5429 - accuracy: 0.8068 - val_loss: 0.6470 - val_accuracy: 0.8294 - lr: 6.2500e-05 Epoch 38/50 60/60 [==============================] - 8s 128ms/step - loss: 0.5511 - accuracy: 0.7965 - val_loss: 0.6126 - val_accuracy: 0.8364 - lr: 3.1250e-05 Epoch 39/50 60/60 [==============================] - 7s 119ms/step - loss: 0.5329 - accuracy: 0.8007 - val_loss: 0.6065 - val_accuracy: 0.8341 - lr: 3.1250e-05 Epoch 40/50 60/60 [==============================] - ETA: 0s - loss: 0.5469 - accuracy: 0.8012 Epoch 40: ReduceLROnPlateau reducing learning rate to 1.5625000742147677e-05. 60/60 [==============================] - 7s 120ms/step - loss: 0.5469 - accuracy: 0.8012 - val_loss: 0.6341 - val_accuracy: 0.8318 - lr: 3.1250e-05 Epoch 41/50 60/60 [==============================] - 7s 118ms/step - loss: 0.5272 - accuracy: 0.8033 - val_loss: 0.6287 - val_accuracy: 0.8318 - lr: 1.5625e-05 Epoch 42/50 60/60 [==============================] - 7s 124ms/step - loss: 0.5644 - accuracy: 0.7933 - val_loss: 0.6249 - val_accuracy: 0.8364 - lr: 1.5625e-05 Epoch 43/50 60/60 [==============================] - ETA: 0s - loss: 0.5440 - accuracy: 0.7975 Epoch 43: ReduceLROnPlateau reducing learning rate to 1e-05. 60/60 [==============================] - 7s 118ms/step - loss: 0.5440 - accuracy: 0.7975 - val_loss: 0.6144 - val_accuracy: 0.8318 - lr: 1.5625e-05 Epoch 44/50 60/60 [==============================] - 8s 130ms/step - loss: 0.5297 - accuracy: 0.8160 - val_loss: 0.6269 - val_accuracy: 0.8364 - lr: 1.0000e-05 Epoch 45/50 60/60 [==============================] - 7s 121ms/step - loss: 0.5337 - accuracy: 0.8023 - val_loss: 0.6153 - val_accuracy: 0.8271 - lr: 1.0000e-05 Epoch 46/50 60/60 [==============================] - 7s 121ms/step - loss: 0.5343 - accuracy: 0.8089 - val_loss: 0.6066 - val_accuracy: 0.8388 - lr: 1.0000e-05 Epoch 47/50 60/60 [==============================] - 7s 117ms/step - loss: 0.5483 - accuracy: 0.7925 - val_loss: 0.6090 - val_accuracy: 0.8388 - lr: 1.0000e-05 Epoch 48/50 60/60 [==============================] - 7s 115ms/step - loss: 0.5557 - accuracy: 0.8033 - val_loss: 0.6213 - val_accuracy: 0.8388 - lr: 1.0000e-05 Epoch 49/50 60/60 [==============================] - 7s 120ms/step - loss: 0.5523 - accuracy: 0.7967 - val_loss: 0.6202 - val_accuracy: 0.8341 - lr: 1.0000e-05 Epoch 50/50 60/60 [==============================] - 7s 117ms/step - loss: 0.5576 - accuracy: 0.7933 - val_loss: 0.6244 - val_accuracy: 0.8388 - lr: 1.0000e-05
In [194]:
# Report the performance of the model.
model_performance_classification_sklearn(model2, X_test_normalized, y_test_original, 'Model 2')
15/15 [==============================] - 0s 19ms/step
Out[194]:
| Model | Accuracy | Recall | Precision | F1 | |
|---|---|---|---|---|---|
| 0 | Model 2 | 0.797895 | 0.797895 | 0.807582 | 0.796945 |
In [199]:
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Model 2 Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations:
- The validation accuracy (83.88%) is higher than the training accuracy (79.33%), which is a positive sign indicating that the model is generalizing well to the validation data. This shows the model is performing much better on the training data than on the validation data.
- The training and validation losses generally decrease over epochs, indicating that the model is learning.
- There is no significant sign of overfitting as the validation accuracy is higher than the training accuracy, and the validation loss does not increase significantly towards the end.
In [196]:
# Evaluate the model on the test data.
accuracy = model2.evaluate(X_test_normalized, y_test_encoded, verbose=2)
15/15 - 0s - loss: 0.7277 - accuracy: 0.7979 - 253ms/epoch - 17ms/step
Observations:
- The model achieved a loss of 0.7277 and an accuracy of 79.79% on the test data.
- The accuracy indicates that the model performs reasonably well, correctly predicting about 80% of the test samples.
In [197]:
# Obtain the output probabilities.
y_pred=model2.predict(X_test_normalized)
15/15 [==============================] - 0s 18ms/step
In [198]:
# Obtaining the categorical values from y_test_encoded and y_pred.
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Model 2 Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()
In [200]:
# Plotting the classification report
print("Model 2 Classification Report")
cr=metrics.classification_report(y_pred_arg, y_test_arg, zero_division=1)
print(cr)
Model 2 Classification Report
precision recall f1-score support
0 0.50 0.52 0.51 25
1 0.90 0.90 0.90 39
2 0.66 0.79 0.72 24
3 0.92 0.92 0.92 61
4 0.55 0.92 0.69 13
5 0.85 0.79 0.82 52
6 0.75 0.71 0.73 69
7 0.91 0.65 0.75 31
8 0.90 0.73 0.81 64
9 0.70 0.84 0.76 19
10 0.84 0.93 0.88 45
11 0.76 0.88 0.82 33
accuracy 0.80 475
macro avg 0.77 0.80 0.78 475
weighted avg 0.81 0.80 0.80 475
Obversations
- The model performs well overall with an accuracy of 80%.
In [212]:
# Define a list of models and their names.
models = [
(model1, 'Model 1'),
(model2, 'Model 2')
]
# Print models performance.
for model, name in models:
print(model_performance_classification_sklearn(model, X_test_normalized, y_test_original, name))
15/15 [==============================] - 0s 16ms/step
Model Accuracy Recall Precision F1
0 Model 1 0.776842 0.776842 0.779542 0.774651
15/15 [==============================] - 0s 10ms/step
Model Accuracy Recall Precision F1
0 Model 2 0.797895 0.797895 0.807582 0.796945
Building Model 3¶
Data Augmentation¶
- Horizontal flip
- Vertical Flip
- Height Shift
- Width shift
- Rotation
- Zoom
- Fill
In [202]:
train_datagen = ImageDataGenerator(
horizontal_flip = True,
vertical_flip = True,
height_shift_range= 0.1,
width_shift_range=0.1,
rotation_range=20,
shear_range = 0.1,
zoom_range=0.1,
fill_mode='nearest'
)
In [203]:
# Clear backend session.
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
In [205]:
vgg_model = VGG16(weights='imagenet', include_top = False, input_shape = (64,64,3))
vgg_model.summary()
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/vgg16/vgg16_weights_tf_dim_ordering_tf_kernels_notop.h5
58889256/58889256 [==============================] - 2s 0us/step
Model: "vgg16"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 64, 64, 3)] 0
block1_conv1 (Conv2D) (None, 64, 64, 64) 1792
block1_conv2 (Conv2D) (None, 64, 64, 64) 36928
block1_pool (MaxPooling2D) (None, 32, 32, 64) 0
block2_conv1 (Conv2D) (None, 32, 32, 128) 73856
block2_conv2 (Conv2D) (None, 32, 32, 128) 147584
block2_pool (MaxPooling2D) (None, 16, 16, 128) 0
block3_conv1 (Conv2D) (None, 16, 16, 256) 295168
block3_conv2 (Conv2D) (None, 16, 16, 256) 590080
block3_conv3 (Conv2D) (None, 16, 16, 256) 590080
block3_pool (MaxPooling2D) (None, 8, 8, 256) 0
block4_conv1 (Conv2D) (None, 8, 8, 512) 1180160
block4_conv2 (Conv2D) (None, 8, 8, 512) 2359808
block4_conv3 (Conv2D) (None, 8, 8, 512) 2359808
block4_pool (MaxPooling2D) (None, 4, 4, 512) 0
block5_conv1 (Conv2D) (None, 4, 4, 512) 2359808
block5_conv2 (Conv2D) (None, 4, 4, 512) 2359808
block5_conv3 (Conv2D) (None, 4, 4, 512) 2359808
block5_pool (MaxPooling2D) (None, 2, 2, 512) 0
=================================================================
Total params: 14714688 (56.13 MB)
Trainable params: 14714688 (56.13 MB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________
In [206]:
# Making all the layers of the VGG model non-trainable, freezing the weights.
for layer in vgg_model.layers:
layer.trainable = False
In [207]:
# Intializing a sequential model
model3 = Sequential()
model3.add(vgg_model)
model3.add(Flatten())
model3.add(Dense(16, activation='relu'))
model3.add(Dropout(0.3))
model3.add(Dense(12, activation='softmax'))
opt=Adam()
model3.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
model3.summary()
WARNING:absl:At this time, the v2.11+ optimizer `tf.keras.optimizers.Adam` runs slowly on M1/M2 Macs, please use the legacy Keras optimizer instead, located at `tf.keras.optimizers.legacy.Adam`.
Model: "sequential_1"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
vgg16 (Functional) (None, 2, 2, 512) 14714688
flatten_1 (Flatten) (None, 2048) 0
dense_2 (Dense) (None, 16) 32784
dropout_1 (Dropout) (None, 16) 0
dense_3 (Dense) (None, 12) 204
=================================================================
Total params: 14747676 (56.26 MB)
Trainable params: 32988 (128.86 KB)
Non-trainable params: 14714688 (56.13 MB)
_________________________________________________________________
In [208]:
# Complete the code to fit the model on train data.
epochs = 50
batch_size = 64
history = model3.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
batch_size=batch_size,
shuffle=False),
epochs=epochs,
steps_per_epoch=X_train_normalized.shape[0] // batch_size,
validation_data=(X_val_normalized,y_val_encoded),
verbose=1,callbacks=[learning_rate_reduction])
Epoch 1/50 60/60 [==============================] - 31s 518ms/step - loss: 2.4355 - accuracy: 0.1377 - val_loss: 2.3486 - val_accuracy: 0.2126 - lr: 0.0010 Epoch 2/50 60/60 [==============================] - 33s 546ms/step - loss: 2.3361 - accuracy: 0.1927 - val_loss: 2.2588 - val_accuracy: 0.2850 - lr: 0.0010 Epoch 3/50 60/60 [==============================] - 35s 582ms/step - loss: 2.2666 - accuracy: 0.2131 - val_loss: 2.1742 - val_accuracy: 0.3014 - lr: 0.0010 Epoch 4/50 60/60 [==============================] - 46s 775ms/step - loss: 2.1986 - accuracy: 0.2334 - val_loss: 2.0840 - val_accuracy: 0.3178 - lr: 0.0010 Epoch 5/50 60/60 [==============================] - 67s 1s/step - loss: 2.1165 - accuracy: 0.2635 - val_loss: 1.9994 - val_accuracy: 0.3411 - lr: 0.0010 Epoch 6/50 60/60 [==============================] - 52s 874ms/step - loss: 2.0600 - accuracy: 0.2760 - val_loss: 1.9618 - val_accuracy: 0.3388 - lr: 0.0010 Epoch 7/50 60/60 [==============================] - 70s 1s/step - loss: 2.0005 - accuracy: 0.2805 - val_loss: 1.9090 - val_accuracy: 0.3528 - lr: 0.0010 Epoch 8/50 60/60 [==============================] - 56s 922ms/step - loss: 1.9711 - accuracy: 0.2834 - val_loss: 1.8734 - val_accuracy: 0.3528 - lr: 0.0010 Epoch 9/50 60/60 [==============================] - 52s 868ms/step - loss: 1.9502 - accuracy: 0.2937 - val_loss: 1.8459 - val_accuracy: 0.3364 - lr: 0.0010 Epoch 10/50 60/60 [==============================] - 59s 993ms/step - loss: 1.9145 - accuracy: 0.2945 - val_loss: 1.8111 - val_accuracy: 0.3621 - lr: 0.0010 Epoch 11/50 60/60 [==============================] - 54s 903ms/step - loss: 1.9005 - accuracy: 0.2998 - val_loss: 1.7753 - val_accuracy: 0.3645 - lr: 0.0010 Epoch 12/50 60/60 [==============================] - 62s 1s/step - loss: 1.8967 - accuracy: 0.3011 - val_loss: 1.7662 - val_accuracy: 0.3855 - lr: 0.0010 Epoch 13/50 60/60 [==============================] - 63s 1s/step - loss: 1.8619 - accuracy: 0.3172 - val_loss: 1.7569 - val_accuracy: 0.3762 - lr: 0.0010 Epoch 14/50 60/60 [==============================] - 60s 997ms/step - loss: 1.8552 - accuracy: 0.3085 - val_loss: 1.7346 - val_accuracy: 0.4042 - lr: 0.0010 Epoch 15/50 60/60 [==============================] - 61s 1s/step - loss: 1.8466 - accuracy: 0.3177 - val_loss: 1.7259 - val_accuracy: 0.4112 - lr: 0.0010 Epoch 16/50 60/60 [==============================] - 64s 1s/step - loss: 1.8339 - accuracy: 0.3241 - val_loss: 1.6941 - val_accuracy: 0.4089 - lr: 0.0010 Epoch 17/50 60/60 [==============================] - 58s 966ms/step - loss: 1.8175 - accuracy: 0.3238 - val_loss: 1.6708 - val_accuracy: 0.4252 - lr: 0.0010 Epoch 18/50 60/60 [==============================] - 57s 957ms/step - loss: 1.8157 - accuracy: 0.3315 - val_loss: 1.6809 - val_accuracy: 0.4276 - lr: 0.0010 Epoch 19/50 60/60 [==============================] - 69s 1s/step - loss: 1.8049 - accuracy: 0.3378 - val_loss: 1.6771 - val_accuracy: 0.4276 - lr: 0.0010 Epoch 20/50 60/60 [==============================] - 59s 979ms/step - loss: 1.7890 - accuracy: 0.3455 - val_loss: 1.6475 - val_accuracy: 0.4486 - lr: 0.0010 Epoch 21/50 60/60 [==============================] - 58s 976ms/step - loss: 1.7877 - accuracy: 0.3476 - val_loss: 1.6533 - val_accuracy: 0.4486 - lr: 0.0010 Epoch 22/50 60/60 [==============================] - 71s 1s/step - loss: 1.7800 - accuracy: 0.3399 - val_loss: 1.6275 - val_accuracy: 0.4603 - lr: 0.0010 Epoch 23/50 60/60 [==============================] - 60s 999ms/step - loss: 1.7674 - accuracy: 0.3460 - val_loss: 1.6136 - val_accuracy: 0.4556 - lr: 0.0010 Epoch 24/50 60/60 [==============================] - 61s 1s/step - loss: 1.7703 - accuracy: 0.3484 - val_loss: 1.6281 - val_accuracy: 0.4369 - lr: 0.0010 Epoch 25/50 60/60 [==============================] - ETA: 0s - loss: 1.7435 - accuracy: 0.3439 Epoch 25: ReduceLROnPlateau reducing learning rate to 0.0005000000237487257. 60/60 [==============================] - 70s 1s/step - loss: 1.7435 - accuracy: 0.3439 - val_loss: 1.6017 - val_accuracy: 0.4416 - lr: 0.0010 Epoch 26/50 60/60 [==============================] - 59s 989ms/step - loss: 1.7588 - accuracy: 0.3436 - val_loss: 1.5955 - val_accuracy: 0.4486 - lr: 5.0000e-04 Epoch 27/50 60/60 [==============================] - 70s 1s/step - loss: 1.7426 - accuracy: 0.3569 - val_loss: 1.5847 - val_accuracy: 0.4509 - lr: 5.0000e-04 Epoch 28/50 60/60 [==============================] - ETA: 0s - loss: 1.7435 - accuracy: 0.3545 Epoch 28: ReduceLROnPlateau reducing learning rate to 0.0002500000118743628. 60/60 [==============================] - 72s 1s/step - loss: 1.7435 - accuracy: 0.3545 - val_loss: 1.5839 - val_accuracy: 0.4509 - lr: 5.0000e-04 Epoch 29/50 60/60 [==============================] - 65s 1s/step - loss: 1.7325 - accuracy: 0.3524 - val_loss: 1.5848 - val_accuracy: 0.4533 - lr: 2.5000e-04 Epoch 30/50 60/60 [==============================] - 75s 1s/step - loss: 1.7252 - accuracy: 0.3598 - val_loss: 1.5835 - val_accuracy: 0.4556 - lr: 2.5000e-04 Epoch 31/50 60/60 [==============================] - ETA: 0s - loss: 1.7445 - accuracy: 0.3505 Epoch 31: ReduceLROnPlateau reducing learning rate to 0.0001250000059371814. 60/60 [==============================] - 65s 1s/step - loss: 1.7445 - accuracy: 0.3505 - val_loss: 1.5948 - val_accuracy: 0.4533 - lr: 2.5000e-04 Epoch 32/50 60/60 [==============================] - 75s 1s/step - loss: 1.7320 - accuracy: 0.3624 - val_loss: 1.5762 - val_accuracy: 0.4650 - lr: 1.2500e-04 Epoch 33/50 60/60 [==============================] - 76s 1s/step - loss: 1.7277 - accuracy: 0.3566 - val_loss: 1.5763 - val_accuracy: 0.4533 - lr: 1.2500e-04 Epoch 34/50 60/60 [==============================] - 64s 1s/step - loss: 1.7366 - accuracy: 0.3627 - val_loss: 1.5731 - val_accuracy: 0.4626 - lr: 1.2500e-04 Epoch 35/50 60/60 [==============================] - ETA: 0s - loss: 1.7344 - accuracy: 0.3558 Epoch 35: ReduceLROnPlateau reducing learning rate to 6.25000029685907e-05. 60/60 [==============================] - 75s 1s/step - loss: 1.7344 - accuracy: 0.3558 - val_loss: 1.5807 - val_accuracy: 0.4579 - lr: 1.2500e-04 Epoch 36/50 60/60 [==============================] - 67s 1s/step - loss: 1.7393 - accuracy: 0.3582 - val_loss: 1.5778 - val_accuracy: 0.4603 - lr: 6.2500e-05 Epoch 37/50 60/60 [==============================] - 71s 1s/step - loss: 1.7372 - accuracy: 0.3550 - val_loss: 1.5751 - val_accuracy: 0.4556 - lr: 6.2500e-05 Epoch 38/50 60/60 [==============================] - ETA: 0s - loss: 1.7315 - accuracy: 0.3611 Epoch 38: ReduceLROnPlateau reducing learning rate to 3.125000148429535e-05. 60/60 [==============================] - 82s 1s/step - loss: 1.7315 - accuracy: 0.3611 - val_loss: 1.5787 - val_accuracy: 0.4556 - lr: 6.2500e-05 Epoch 39/50 60/60 [==============================] - 64s 1s/step - loss: 1.7395 - accuracy: 0.3476 - val_loss: 1.5744 - val_accuracy: 0.4579 - lr: 3.1250e-05 Epoch 40/50 60/60 [==============================] - 73s 1s/step - loss: 1.7440 - accuracy: 0.3500 - val_loss: 1.5742 - val_accuracy: 0.4603 - lr: 3.1250e-05 Epoch 41/50 60/60 [==============================] - ETA: 0s - loss: 1.7423 - accuracy: 0.3471 Epoch 41: ReduceLROnPlateau reducing learning rate to 1.5625000742147677e-05. 60/60 [==============================] - 68s 1s/step - loss: 1.7423 - accuracy: 0.3471 - val_loss: 1.5772 - val_accuracy: 0.4556 - lr: 3.1250e-05 Epoch 42/50 60/60 [==============================] - 71s 1s/step - loss: 1.7407 - accuracy: 0.3505 - val_loss: 1.5773 - val_accuracy: 0.4556 - lr: 1.5625e-05 Epoch 43/50 60/60 [==============================] - 75s 1s/step - loss: 1.7489 - accuracy: 0.3471 - val_loss: 1.5767 - val_accuracy: 0.4556 - lr: 1.5625e-05 Epoch 44/50 60/60 [==============================] - ETA: 0s - loss: 1.7393 - accuracy: 0.3500 Epoch 44: ReduceLROnPlateau reducing learning rate to 1e-05. 60/60 [==============================] - 74s 1s/step - loss: 1.7393 - accuracy: 0.3500 - val_loss: 1.5766 - val_accuracy: 0.4579 - lr: 1.5625e-05 Epoch 45/50 60/60 [==============================] - 82s 1s/step - loss: 1.7392 - accuracy: 0.3574 - val_loss: 1.5765 - val_accuracy: 0.4556 - lr: 1.0000e-05 Epoch 46/50 60/60 [==============================] - 81s 1s/step - loss: 1.7252 - accuracy: 0.3619 - val_loss: 1.5764 - val_accuracy: 0.4556 - lr: 1.0000e-05 Epoch 47/50 60/60 [==============================] - 70s 1s/step - loss: 1.7422 - accuracy: 0.3577 - val_loss: 1.5761 - val_accuracy: 0.4579 - lr: 1.0000e-05 Epoch 48/50 60/60 [==============================] - 71s 1s/step - loss: 1.7267 - accuracy: 0.3606 - val_loss: 1.5766 - val_accuracy: 0.4556 - lr: 1.0000e-05 Epoch 49/50 60/60 [==============================] - 74s 1s/step - loss: 1.7357 - accuracy: 0.3611 - val_loss: 1.5757 - val_accuracy: 0.4579 - lr: 1.0000e-05 Epoch 50/50 60/60 [==============================] - 83s 1s/step - loss: 1.7375 - accuracy: 0.3619 - val_loss: 1.5754 - val_accuracy: 0.4579 - lr: 1.0000e-05
In [209]:
# Report the performance of the model.
model_performance_classification_sklearn(model3, X_test_normalized, y_test_original, 'Model 3')
15/15 [==============================] - 5s 352ms/step
Out[209]:
| Model | Accuracy | Recall | Precision | F1 | |
|---|---|---|---|---|---|
| 0 | Model 3 | 0.435789 | 0.435789 | 0.319426 | 0.35037 |
In [213]:
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('Model 3 Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
Observations:
- Model performs poorly across all results. Low accuracy, recall, precision and F1 score.
In [214]:
# Evaluate the model on the test data.
accuracy = model3.evaluate(X_test_normalized, y_test_encoded, verbose=2)
15/15 - 4s - loss: 1.6067 - accuracy: 0.4358 - 4s/epoch - 237ms/step
Observations:
- The model achieved a loss of 1.6067 and an accuracy of 43.58% on the test data.
- The accuracy indicates that the model performs poorly.
In [215]:
# Obtain the output probabilities.
y_pred=model3.predict(X_test_normalized)
15/15 [==============================] - 4s 234ms/step
In [216]:
# Obtaining the categorical values from y_test_encoded and y_pred.
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Model 3 Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()
In [217]:
# Plotting the classification report.
cr=metrics.classification_report(y_pred_arg, y_test_arg, zero_division=1)
print(cr)
precision recall f1-score support
0 0.00 1.00 0.00 0
1 0.36 0.32 0.34 44
2 0.28 0.57 0.37 14
3 0.80 0.51 0.62 97
4 0.00 1.00 0.00 0
5 0.33 0.40 0.36 40
6 0.98 0.49 0.65 131
7 0.00 1.00 0.00 0
8 0.37 0.45 0.40 42
9 0.00 1.00 0.00 0
10 0.74 0.35 0.47 106
11 0.00 0.00 0.00 1
accuracy 0.44 475
macro avg 0.32 0.59 0.27 475
weighted avg 0.70 0.44 0.52 475
Obversations
- The model performs poorly overall with an accuracy of 44%.
- Model is underfitting based on the low overall accuracy, low macro and weighted average F1-scores.
- The model appears to be too simple to capture the underlying patterns in the data, leading to poor performance on the test set.
In [218]:
# Define a list of models and their names.
models = [
(model1, 'Model 1'),
(model2, 'Model 2'),
(model3, 'Model 3')
]
# Print models performance.
for model, name in models:
print(model_performance_classification_sklearn(model, X_test_normalized, y_test_original, name))
15/15 [==============================] - 0s 20ms/step
Model Accuracy Recall Precision F1
0 Model 1 0.776842 0.776842 0.779542 0.774651
15/15 [==============================] - 0s 10ms/step
Model Accuracy Recall Precision F1
0 Model 2 0.797895 0.797895 0.807582 0.796945
15/15 [==============================] - 3s 222ms/step
Model Accuracy Recall Precision F1
0 Model 3 0.435789 0.435789 0.319426 0.35037
Build Model 4¶
Data Augmentation¶
- Transfer learning from MobileNet.
In [219]:
# Clear backend session.
backend.clear_session()
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
In [220]:
# Use MobileNet CNN.
image_input=Input(shape=(64,64,3))
base_model = MobileNet(input_tensor=image_input,weights='imagenet', include_top=False)
x = base_model.output
x = Flatten()(x)
x = Dense(64, activation='relu')(x)
x = Dropout(0.3)(x)
x = Dense(32, activation='relu')(x)
predictions = Dense(12, activation='softmax')(x)
model4 = Model(inputs=base_model.input, outputs=predictions)
model4.compile(optimizer=Adam(learning_rate=1e-4), loss='categorical_crossentropy',metrics=['accuracy'])
model4.summary()
WARNING:tensorflow:`input_shape` is undefined or non-square, or `rows` is not in [128, 160, 192, 224]. Weights for input shape (224, 224) will be loaded as the default.
WARNING:tensorflow:`input_shape` is undefined or non-square, or `rows` is not in [128, 160, 192, 224]. Weights for input shape (224, 224) will be loaded as the default.
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/mobilenet/mobilenet_1_0_224_tf_no_top.h5 17225924/17225924 [==============================] - 0s 0us/step
WARNING:absl:At this time, the v2.11+ optimizer `tf.keras.optimizers.Adam` runs slowly on M1/M2 Macs, please use the legacy Keras optimizer instead, located at `tf.keras.optimizers.legacy.Adam`.
Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 64, 64, 3)] 0
conv1 (Conv2D) (None, 32, 32, 32) 864
conv1_bn (BatchNormalizati (None, 32, 32, 32) 128
on)
conv1_relu (ReLU) (None, 32, 32, 32) 0
conv_dw_1 (DepthwiseConv2D (None, 32, 32, 32) 288
)
conv_dw_1_bn (BatchNormali (None, 32, 32, 32) 128
zation)
conv_dw_1_relu (ReLU) (None, 32, 32, 32) 0
conv_pw_1 (Conv2D) (None, 32, 32, 64) 2048
conv_pw_1_bn (BatchNormali (None, 32, 32, 64) 256
zation)
conv_pw_1_relu (ReLU) (None, 32, 32, 64) 0
conv_pad_2 (ZeroPadding2D) (None, 33, 33, 64) 0
conv_dw_2 (DepthwiseConv2D (None, 16, 16, 64) 576
)
conv_dw_2_bn (BatchNormali (None, 16, 16, 64) 256
zation)
conv_dw_2_relu (ReLU) (None, 16, 16, 64) 0
conv_pw_2 (Conv2D) (None, 16, 16, 128) 8192
conv_pw_2_bn (BatchNormali (None, 16, 16, 128) 512
zation)
conv_pw_2_relu (ReLU) (None, 16, 16, 128) 0
conv_dw_3 (DepthwiseConv2D (None, 16, 16, 128) 1152
)
conv_dw_3_bn (BatchNormali (None, 16, 16, 128) 512
zation)
conv_dw_3_relu (ReLU) (None, 16, 16, 128) 0
conv_pw_3 (Conv2D) (None, 16, 16, 128) 16384
conv_pw_3_bn (BatchNormali (None, 16, 16, 128) 512
zation)
conv_pw_3_relu (ReLU) (None, 16, 16, 128) 0
conv_pad_4 (ZeroPadding2D) (None, 17, 17, 128) 0
conv_dw_4 (DepthwiseConv2D (None, 8, 8, 128) 1152
)
conv_dw_4_bn (BatchNormali (None, 8, 8, 128) 512
zation)
conv_dw_4_relu (ReLU) (None, 8, 8, 128) 0
conv_pw_4 (Conv2D) (None, 8, 8, 256) 32768
conv_pw_4_bn (BatchNormali (None, 8, 8, 256) 1024
zation)
conv_pw_4_relu (ReLU) (None, 8, 8, 256) 0
conv_dw_5 (DepthwiseConv2D (None, 8, 8, 256) 2304
)
conv_dw_5_bn (BatchNormali (None, 8, 8, 256) 1024
zation)
conv_dw_5_relu (ReLU) (None, 8, 8, 256) 0
conv_pw_5 (Conv2D) (None, 8, 8, 256) 65536
conv_pw_5_bn (BatchNormali (None, 8, 8, 256) 1024
zation)
conv_pw_5_relu (ReLU) (None, 8, 8, 256) 0
conv_pad_6 (ZeroPadding2D) (None, 9, 9, 256) 0
conv_dw_6 (DepthwiseConv2D (None, 4, 4, 256) 2304
)
conv_dw_6_bn (BatchNormali (None, 4, 4, 256) 1024
zation)
conv_dw_6_relu (ReLU) (None, 4, 4, 256) 0
conv_pw_6 (Conv2D) (None, 4, 4, 512) 131072
conv_pw_6_bn (BatchNormali (None, 4, 4, 512) 2048
zation)
conv_pw_6_relu (ReLU) (None, 4, 4, 512) 0
conv_dw_7 (DepthwiseConv2D (None, 4, 4, 512) 4608
)
conv_dw_7_bn (BatchNormali (None, 4, 4, 512) 2048
zation)
conv_dw_7_relu (ReLU) (None, 4, 4, 512) 0
conv_pw_7 (Conv2D) (None, 4, 4, 512) 262144
conv_pw_7_bn (BatchNormali (None, 4, 4, 512) 2048
zation)
conv_pw_7_relu (ReLU) (None, 4, 4, 512) 0
conv_dw_8 (DepthwiseConv2D (None, 4, 4, 512) 4608
)
conv_dw_8_bn (BatchNormali (None, 4, 4, 512) 2048
zation)
conv_dw_8_relu (ReLU) (None, 4, 4, 512) 0
conv_pw_8 (Conv2D) (None, 4, 4, 512) 262144
conv_pw_8_bn (BatchNormali (None, 4, 4, 512) 2048
zation)
conv_pw_8_relu (ReLU) (None, 4, 4, 512) 0
conv_dw_9 (DepthwiseConv2D (None, 4, 4, 512) 4608
)
conv_dw_9_bn (BatchNormali (None, 4, 4, 512) 2048
zation)
conv_dw_9_relu (ReLU) (None, 4, 4, 512) 0
conv_pw_9 (Conv2D) (None, 4, 4, 512) 262144
conv_pw_9_bn (BatchNormali (None, 4, 4, 512) 2048
zation)
conv_pw_9_relu (ReLU) (None, 4, 4, 512) 0
conv_dw_10 (DepthwiseConv2 (None, 4, 4, 512) 4608
D)
conv_dw_10_bn (BatchNormal (None, 4, 4, 512) 2048
ization)
conv_dw_10_relu (ReLU) (None, 4, 4, 512) 0
conv_pw_10 (Conv2D) (None, 4, 4, 512) 262144
conv_pw_10_bn (BatchNormal (None, 4, 4, 512) 2048
ization)
conv_pw_10_relu (ReLU) (None, 4, 4, 512) 0
conv_dw_11 (DepthwiseConv2 (None, 4, 4, 512) 4608
D)
conv_dw_11_bn (BatchNormal (None, 4, 4, 512) 2048
ization)
conv_dw_11_relu (ReLU) (None, 4, 4, 512) 0
conv_pw_11 (Conv2D) (None, 4, 4, 512) 262144
conv_pw_11_bn (BatchNormal (None, 4, 4, 512) 2048
ization)
conv_pw_11_relu (ReLU) (None, 4, 4, 512) 0
conv_pad_12 (ZeroPadding2D (None, 5, 5, 512) 0
)
conv_dw_12 (DepthwiseConv2 (None, 2, 2, 512) 4608
D)
conv_dw_12_bn (BatchNormal (None, 2, 2, 512) 2048
ization)
conv_dw_12_relu (ReLU) (None, 2, 2, 512) 0
conv_pw_12 (Conv2D) (None, 2, 2, 1024) 524288
conv_pw_12_bn (BatchNormal (None, 2, 2, 1024) 4096
ization)
conv_pw_12_relu (ReLU) (None, 2, 2, 1024) 0
conv_dw_13 (DepthwiseConv2 (None, 2, 2, 1024) 9216
D)
conv_dw_13_bn (BatchNormal (None, 2, 2, 1024) 4096
ization)
conv_dw_13_relu (ReLU) (None, 2, 2, 1024) 0
conv_pw_13 (Conv2D) (None, 2, 2, 1024) 1048576
conv_pw_13_bn (BatchNormal (None, 2, 2, 1024) 4096
ization)
conv_pw_13_relu (ReLU) (None, 2, 2, 1024) 0
flatten (Flatten) (None, 4096) 0
dense (Dense) (None, 64) 262208
dropout (Dropout) (None, 64) 0
dense_1 (Dense) (None, 32) 2080
dense_2 (Dense) (None, 12) 396
=================================================================
Total params: 3493548 (13.33 MB)
Trainable params: 3471660 (13.24 MB)
Non-trainable params: 21888 (85.50 KB)
_________________________________________________________________
In [222]:
# Train the model on the new data.
batch_size = 32
annealer = LearningRateScheduler(lambda x: 1e-3 * 0.8 ** x)
earlystop = EarlyStopping(patience=5)
history4 = model4.fit(
X_train_normalized, y_train_encoded, batch_size=batch_size,
epochs=50,
validation_data=(X_val_normalized, y_val_encoded),
callbacks=[annealer, earlystop]
)
Epoch 1/50 121/121 [==============================] - 11s 92ms/step - loss: 0.8784 - accuracy: 0.7317 - val_loss: 1.0041 - val_accuracy: 0.7313 - lr: 0.0010 Epoch 2/50 121/121 [==============================] - 11s 92ms/step - loss: 0.6528 - accuracy: 0.8045 - val_loss: 1.0133 - val_accuracy: 0.7500 - lr: 8.0000e-04 Epoch 3/50 121/121 [==============================] - 12s 95ms/step - loss: 0.5343 - accuracy: 0.8349 - val_loss: 0.8932 - val_accuracy: 0.7687 - lr: 6.4000e-04 Epoch 4/50 121/121 [==============================] - 12s 98ms/step - loss: 0.4110 - accuracy: 0.8653 - val_loss: 0.6760 - val_accuracy: 0.8341 - lr: 5.1200e-04 Epoch 5/50 121/121 [==============================] - 12s 103ms/step - loss: 0.4416 - accuracy: 0.8679 - val_loss: 0.5054 - val_accuracy: 0.8645 - lr: 4.0960e-04 Epoch 6/50 121/121 [==============================] - 15s 124ms/step - loss: 0.2846 - accuracy: 0.9002 - val_loss: 0.3972 - val_accuracy: 0.8855 - lr: 3.2768e-04 Epoch 7/50 121/121 [==============================] - 22s 179ms/step - loss: 0.2471 - accuracy: 0.9194 - val_loss: 0.3725 - val_accuracy: 0.9089 - lr: 2.6214e-04 Epoch 8/50 121/121 [==============================] - 25s 205ms/step - loss: 0.1862 - accuracy: 0.9306 - val_loss: 0.3679 - val_accuracy: 0.8855 - lr: 2.0972e-04 Epoch 9/50 121/121 [==============================] - 25s 207ms/step - loss: 0.1874 - accuracy: 0.9389 - val_loss: 0.4274 - val_accuracy: 0.8949 - lr: 1.6777e-04 Epoch 10/50 121/121 [==============================] - 19s 156ms/step - loss: 0.1325 - accuracy: 0.9506 - val_loss: 0.4217 - val_accuracy: 0.8995 - lr: 1.3422e-04 Epoch 11/50 121/121 [==============================] - 18s 146ms/step - loss: 0.1406 - accuracy: 0.9519 - val_loss: 0.4588 - val_accuracy: 0.8949 - lr: 1.0737e-04 Epoch 12/50 121/121 [==============================] - 18s 145ms/step - loss: 0.1230 - accuracy: 0.9592 - val_loss: 0.4271 - val_accuracy: 0.9065 - lr: 8.5899e-05 Epoch 13/50 121/121 [==============================] - 18s 146ms/step - loss: 0.0993 - accuracy: 0.9662 - val_loss: 0.4475 - val_accuracy: 0.9065 - lr: 6.8719e-05
In [223]:
# Report the performance of the model.
model_performance_classification_sklearn(model4, X_test_normalized, y_test_original, 'Model 4')
15/15 [==============================] - 1s 20ms/step
Out[223]:
| Model | Accuracy | Recall | Precision | F1 | |
|---|---|---|---|---|---|
| 0 | Model 4 | 0.896842 | 0.896842 | 0.898085 | 0.896296 |
In [224]:
plt.plot(history4.history['accuracy'])
plt.plot(history4.history['val_accuracy'])
plt.title('Model 4 Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
In [225]:
# Evaluate the model on the test data.
accuracy = model4.evaluate(X_test_normalized, y_test_encoded, verbose=2)
15/15 - 0s - loss: 0.4112 - accuracy: 0.8968 - 408ms/epoch - 27ms/step
Observations:
- The model achieved a loss of 0.4112 and an accuracy of 89.68% on the test data.
- The accuracy indicates that the model performs well.
In [226]:
# Obtain the output probabilities.
y_pred=model4.predict(X_test_normalized)
15/15 [==============================] - 1s 27ms/step
In [227]:
# Obtaining the categorical values from y_test_encoded and y_pred.
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)
# Plotting the Confusion Matrix using confusion matrix() function which is also predefined in tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(12, 12))
sns.heatmap(
confusion_matrix,
annot=True,
linewidths=.4,
fmt="d",
square=True,
ax=ax
)
# Setting the labels to both the axes
ax.set_xlabel('Predicted labels');ax.set_ylabel('True labels');
ax.set_title('Model 4 Confusion Matrix');
ax.xaxis.set_ticklabels(list(enc.classes_),rotation=40)
ax.yaxis.set_ticklabels(list(enc.classes_),rotation=20)
plt.show()
In [228]:
# Plotting the classification report
print("Model 4 Classification Report")
cr=metrics.classification_report(y_pred_arg, y_test_arg, zero_division=1)
print(cr)
Model 4 Classification Report
precision recall f1-score support
0 0.46 0.50 0.48 24
1 0.95 1.00 0.97 37
2 0.90 0.96 0.93 27
3 0.95 0.98 0.97 59
4 0.68 0.83 0.75 18
5 0.98 0.94 0.96 50
6 0.89 0.78 0.83 74
7 0.95 0.95 0.95 22
8 0.92 0.89 0.91 54
9 0.87 0.87 0.87 23
10 0.96 1.00 0.98 48
11 0.95 0.92 0.94 39
accuracy 0.90 475
macro avg 0.87 0.89 0.88 475
weighted avg 0.90 0.90 0.90 475
Obversations
- The overall accuracy of the model is 90%. This indicates that the model correctly predicted 90% of the test samples.
- The macro average is the unweighted mean of the precision, recall, and F1-score for all classes. It indicates that, on average, the model has a precision of 87%, recall of 89%, and F1-score of 88%.
In [229]:
# Define a list of models and their names.
models = [
(model1, 'Model 1'),
(model2, 'Model 2'),
(model3, 'Model 3'),
(model4, 'Model 4')
]
# Print models performance.
for model, name in models:
print(model_performance_classification_sklearn(model, X_test_normalized, y_test_original, name))
15/15 [==============================] - 0s 18ms/step
Model Accuracy Recall Precision F1
0 Model 1 0.776842 0.776842 0.779542 0.774651
15/15 [==============================] - 0s 10ms/step
Model Accuracy Recall Precision F1
0 Model 2 0.797895 0.797895 0.807582 0.796945
15/15 [==============================] - 3s 232ms/step
Model Accuracy Recall Precision F1
0 Model 3 0.435789 0.435789 0.319426 0.35037
15/15 [==============================] - 0s 21ms/step
Model Accuracy Recall Precision F1
0 Model 4 0.896842 0.896842 0.898085 0.896296
Visualizing the prediction¶
In [230]:
# Visualizing the predicted and correct label of images from test data
plt.figure(figsize=(2,2))
plt.imshow(X_test[2])
plt.show()
## Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(model4.predict((X_test_normalized[2].reshape(1,64,64,3))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[2]) # using inverse_transform() to get the output label from the output vector
plt.figure(figsize=(2,2))
plt.imshow(X_test[33])
plt.show()
## Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(model4.predict((X_test_normalized[33].reshape(1,64,64,3))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[33]) # using inverse_transform() to get the output label from the output vector
plt.figure(figsize=(2,2))
plt.imshow(X_test[59],)
plt.show()
## Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(model4.predict((X_test_normalized[59].reshape(1,64,64,3))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[59]) # using inverse_transform() to get the output label from the output vector
plt.figure(figsize=(2,2))
plt.imshow(X_test[36])
plt.show()
## Complete the code to predict the test data using the final model selected
print('Predicted Label', enc.inverse_transform(model4.predict((X_test_normalized[36].reshape(1,64,64,3))))) # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[36]) # using inverse_transform() to get the output label from the output vector
1/1 [==============================] - 0s 26ms/step Predicted Label ['Small-flowered Cranesbill'] True Label Small-flowered Cranesbill
1/1 [==============================] - 0s 25ms/step Predicted Label ['Cleavers'] True Label Cleavers
1/1 [==============================] - 0s 25ms/step Predicted Label ['Common Chickweed'] True Label Common Chickweed
1/1 [==============================] - 0s 24ms/step Predicted Label ['Shepherds Purse'] True Label Shepherds Purse
| Model | Accuracy | Recall | Precision | F1 |
|---|---|---|---|---|
| Model 1 | 0.776842 | 0.776842 | 0.779542 | 0.774651 |
| Model 2 | 0.797895 | 0.797895 | 0.807582 | 0.796945 |
| Model 3 | 0.435789 | 0.435789 | 0.319426 | 0.350370 |
| ** Model 4 | 0.896842 | 0.896842 | 0.898085 | 0.896296 |
Model 4 is the chosen model with a recall rate of 90%, precision of 90%, and an F1-score of 90%. This model performs superior compared to other models.
Actionable Insights¶
- Data Augmentation: The original data set was primarily a standard view of a photo. Plant in frame, horizontally viewing and with similar background. All techniques should be used to improve identification. Include rotating, flipping, scaling, colors to help widen the data set features.
- Model Evaluation: Model 4 shows a 90% accuracy which is quite good but there is always room for improvement. With new data augmentation tweaks and a more robust data set contiunal improvements to the model can be made and possibly even lead to a new model selection.
- Ongoing Monitoring: As new data is provided to the system new model results need to be reviewed to ensure the proper training model is being used.
Business Recommendations¶
- Improve Data Collection: Improving any model requires data and this should be the top priority to ensure accurate plant predictions.
- Continuous Model Optimization: The models used should be continually running to improve based on new data collected.
- Create a Mobile App: Develop a mobile app to provide to users to add the data collection and also provide a wider variety of environments, backgrounds, colors, etc. to help improve predictions and handling of noise from the variety of photos.